The present disclosure relates to natural language processing and classification of textual documents and more specifically to assigning a score that reflects the polarity and magnitude of the sentiment expressed
Good automatic classification is challenging: it comes at a high cost (of speed and resources) and often leads to poor results. Human classification by an expert produces much better results, but is often too expensive and slow.
In some applications many of the passages exhibit similarities. For example in a collection of short user posts (such as Tweets or Facebook comments), an original post may lead to very many repetitions with only minor modifications. In this case it is reasonable to use an expensive method of classification (a human, or a high-accuracy automatic process) to classify one of the posts and use a fast automatic method to find all similar posts and classify them automatically, obtaining a compromise in accuracy, speed and cost.
A passage of text may be a few words, a sentence, a paragraph or an entire document. Passages of text are common in the Internet, for example as Tweets, Facebook posts, blog posts or blog comments, etc. There are many reasons to classify passages of text. For example, one may be interested in classifying passages by their topic, as spam or not spam, or by semantic properties of the sentence such as its sentiment or polarity (whether its tone is positive or negative, for example).
Sentiment classification is a method helpful when tracking the overall perception of brands, companies or products. Sentiment classification can be used on the Internet to obtain a measure of the reception of a brand or product on the Internet, for example. Business analytics applications could use sentiment classification over a plethora of textual sources, such as reviews, blogs posts, and information present in Social media online sites such as Facebook or Twitter, in order to quantify the perception of brands.
Sentiment classification pertains to assigning a class between either a discrete set of values (positive, neutral or negative) or a real valued normalized score, where the higher bound represents positive sentiment, and the lower bound negative sentiment. Typically, sentiment classifiers make use of the words in a document to classify a piece of text, in which some words are assumed to convey a particular sentiment value. In order to determine which words are indicative of a particular sentiment class or value, some pieces of text are usually classified by a human, which assigns a label to the text. Further, the label of the piece of text are propagated to the words in the text, and these labels are assigned a real valued weight, derived from the information conveyed by the labels of the whole collection of passages or documents. These weights are further computed using methods derived from statistical learning theory (machine learning).
However, the manual process of assigning the labels to the pieces of text is time consuming and expensive, as it is being dependent on human intervention.
A method to perform automatic sentiment class so is described here, whereby the invention makes use of a nearest-neighbor classifier to locate all the similar passages to a passage of known class.
Nearest neighbor search (NNS), also known as proximity search, similarity search or closest point search, is an optimization problem for finding closest points in metric spaces. The problem is: given a set S of points in a metric space M and a query point q∈M, find the closest point in S to q. In many cases, M is taken to be d-dimensional Euclidean space and distance is measured by Euclidean distance or Manhattan distance, but other spaces and distances can be used. For example in the case of points representing text passages one may use a string metric such as Hamming distance or Levenshtein distance.
There exist a number of methods to perform NNS on a given collection of points and a given metric. Common methods include: linear search, space partitioning, locality sensitive hashing, or methods based on compression or clustering of the points.
Opinion mining and sentiment analysis. Bo Pang and Lillian Lee. Foundations and Trends® in Information Retrieval 2(1-2), pp. 1-135, Now Publishers Inc, 2008, presents and overview of recent sentiment classification methods, with an emphasis on classification features based on combinations of words in the document.
U.S. Publication Number US 2009/0125371 A1, filing date Aug. 23, 2007 (Tyler J. Neylon et al.) describes a domain-specific sentiment classifier that can be used to score the polarity and magnitude of sentiment expressed by domain-specific documents.
U.S. Publication Number US 2010/0150393 A1, filing date Dec. 16, 2008 (Xiaochuan Ni et al.) disclose a system to classify textual data according to their sentiment using domain data.
U.S. Publication Number US 2008/0249764 A1, filing date Dec. 5, 2007 (Shen Huan et al.) describes a system that classifies text according to their sentiment, using complex features such as expressions, negation patterns, sentiment specific sections of a product review and so on.
U.S. Pat. No. 7,788,087, issue date Aug. 31, 2010 (Simon H. Corston-Oliver et al.) describes a system for identifying, extracting, clustering and analyzing sentiment-bearing text.
U.S. Publication Number US 2011/0137906 A1 describes a method for analyzing sentiment, comprising of collecting an object from a external content repository; the collected objects forming a content database and extracting a snippet related to the subject from the content database.